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Video viewing system and method 



The invention relates to a video viewing system that supports time-shift 
viewing of a video stream. 

5 PCX patent application W099/33265 discloses a video viewing system that is 

capable of operating in a number of modes of operation, including a time shift mode, a live 
play mode and a pause mode. In the time-shift mode an incoming video stream is recorded in 
a storage unit and in parallel therewith a significantly older (e.g» ten minutes older) part of the 
incoming stream is replayed to the output of the system fi'om the storage unit In the live play 

10 mode the incoming stream is fed directly to the output without any significant delay. In the 
pause mode the system outputs a stiU image, which is the last image output during preceding 
live play and simultaneously records the incoming video stream in ttie storage unit. 

Video mode switching between the different modes is not described in 
W099/33265. In principle, video mode switching could be achieved by restarting the video 

15 system each time when a video mode switch command is received, starting up data flows as 
required for the relevant mode. This, however, has the disadvantage that an intemqption may 
occur in the video output signal, as a result of the need to fill the various pipe-lines in the 
system and to restart the decoding process, often starting from an arbitrary time point in the 
incoming video stream which does not permit decoding to start immediately. 

20 An alternative is to realize video mode switches by means of multiplexing. In 

this case, the video system keeps all tasks that need to be executed in any one of the modes 
(i.e. at least an input task, a recording task, a replay task and a decoding task) running in all 
modes. One or more multiplexers switch the sources of one or more of the tasks during a 
video mode switch. Because none of the tasks needs to be restarted this potentially eliminates 

25 interruptions. However, it has the disadvantage that certain tasks keep running also when 
they are not needed. In a video system in wliich the tasks are implemented as software tasks 
nmning on a general-purpose processor, the processor resources occupied by the tasks remain 
unavailable to other processes. In a hardware implemented system the hardware that 
performs the tasks continues to consume umiecessary power supply current. 
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European patent application No, 01203905.3 (impublished at the priority date 
of the present plication and assigned to the same assignee) describes a mechanism for 
dynamically reconfiguring networks of processes or tasks. 



5 

Among others, it is an object of the invention to provide for a video display 
system that is capable of operating in several modes of operation including a tune-shift mode, 
wherein video mode switching can be realized without significant inteiruption and a task not 
needed during a certain mode need not continue execution in that certain mode, 

1 0 The video display system according to the invention is set forth in Claim 1 . 

FIFO (First In First Out) communication buffers with reassignable sources and destinations 
are used for commimicating video streams between the input, decoding, recording and replay 
tasks. During a mode switch dynamic reconfiguration is realized by reassigning the soim:e or 
destination of a FIFO communication buffer. The other side of tlie FIFO communication 

15 buffer stays assigned to the same task before and after the mode switch. At the reassigmnent 
video data may still be present in the FIFO communication buffer. The task that stays 
assigned to tiie FIFO communication buffer keeps on executing during the mode switch, as 
£ar as video data is available in the FIFO communication buffer. Thus, the use of FIFO 
conununication buff^ which remain active during the mode switch ensures that mode 

20 switching can be performed without intermption of the video streams and without requiring 
tasks to remain active if they are no longer needed after the mode switch. The resources used 
by such a task may be fireed after the mode switch. 

In an embodiment, dynamic reassigmnent involves detachment and subsequmt 
attachment of respective sources or destinations of video streams to the commimication 

25 biiffer. Detachment and attachment occur in response to receiving a mode-switch command, 
but with delays until a subsequent boimdary between closed Groups of Pictures has been 
detected in tiie detached and attached stream respectively. A closed Group of Pictures in a 
video stream encodes a series of firames of video infonnation that can be decoded without 
requiring access to frames outside the group. By delaying detachment and attachment it is 

30 prevented that "orphaned" video data occius in the communication buffer, that cannot be 
decoded, which might give rise to interruptions in the decoding task. 
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These and other objects and advantageous aspects will be described using the 
following figures. 

Figure 1 shows a video di^lay system 

Figures 2a-c show task graphs during different video modes 



Figure 1 shows a video display system. The system contains a video stream 
input 10, a video stream output 12, a plurality of processing task elements 14a-f, a plurality . 
of FIFO communication buffers 16a-d, a buffer connection element 17, a storage device 18 

10 and a switching control unit 15. A video display device 19 is coupled to output 12. 

Processing task elements 14a-f include an input task element 14a, a recording task element 
14b, a replay task element 14c, a decoder task element 14f plus optional further elements 
14d,e. Input 10 is coupled to input task element 14a. Output 12 is coupled to decoder task 
element 14f Storage device 18 is coupled to recording task element 14b and replay task 

15 element 14c. Processing task elements 14a-f are coupled to FIFO communication buffers 
16a-d via buffer coimection element 17. Switching control unit 15 is coupled to processing 
task elements 14a-f, FIFO communication buffers 16a-d and buffer connection element 17. 

In a first embodiment each component shown in figure 1 is a dedicated 
hardware element. In other embodiments one or more of the components may be 

20 implemented in a general purpose processor by means of suitable computer programs. For 
example in the first embodiment processing task elements 14a-f are each different hardware 
elements with a design dedicated to the task to be performed by the processing task element 
14a-f, but in the other embodiments part or all of the processing task elements 14a-f may be 
implemented as different computer programs loaded in a general purpose processor. 

25 Similarly, in the first embodiment FIFO communication buffers 16a-d are hardware FIFO 
buffers with data inputs, data outputs, and buffer fiill and empty signaling outputs. However, 
in the further embodiments FIFO communication buffers 16a-d may be implemented as 
different buffer areas in a processor memory with appropriate software for signaling to task 
elements 14a-f whether or not the buffers are full or whether or not they are empty. Also 

30 similarly, in the first embodiment buffer connection element 17 may be a hardware element 
such as a switch matrix and in the further embodiments buffer coruiection element 17 may be 
implemented an interface program between FIFO communication buffers 16a-d and 
processing task elements 14a-f Furthermore switching control imit 15 may be implemented 
in hardware or as a software programmed computer. 
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In operation an incoming video stream is applied to input 1 0 and an outgoing 
video stream is produced at output 12, for display at video display device 19. In between, 
processing task elements perform various processing tasks on video data that derives from 
the incoming stream. 

5 Video mode control unit 15 controls the video mode in which the video 

viewing system operates. The combination of tasks executed depends on the video mode in 
which the system operates* The video modes include a live play mode, a pause mode and a 
time-shift mode. 

Figures 2a-c show task graphs of the processing in different video modes. 

10 Figure 2a shows operation in the live play mode, in an embodiment where the incoming 

stream is an analog video signal. In this case an input task 20 is performed which receives tlie 
incoming stream and outputs an encoded stream to a decoding task 22 via a first connection 
21. Decoding task 22 outputs a decoded stream for use by video display device 19 (not 
shown). Although a single decoding task 22 has been shown it will be appreciated that the 

15 decoding task may in fact be comprised of several tasks, such as a demultiplexing task for 
demultiplexing a program from an MPEG transport stream, followed by a program decoding 
task operating on the demxiltiplexed program* Similarly separate video and audio stream 
processing tasks may occur. For reasons of simplicity only a single task is shown. 

Figure 2b shows operation in the pause mode. In this mode a recorder task 24 

20 has been added. Input task 20 continues to be performed and outputs the encoded stream to 
recorder task 24 via first connection 21 . Recorder task 24 records the encoded stream in 
storage device 18 (not shown). Decoder task 22 has been switched to a "freeze" state, in 
which it permanently outputs its last received video frame, without reading from its input. Its 
input connection has been uncoupled. In an alternative embodiment, in which decoder task 

25 22 is arranged to permanently output its last full received frame when its finds that no new 
frame has been supplied to its input, first connection 21 may be left connected to decoder 
task 22, a second comiection being added between input task 20 and recording task 24. In this 
alternative embodiment, first connection 21 may be disconnected from input task 20. 

Figure 2c shows operation in the tune-shift mode. In this mode the input task 

30 20, the decoder task 22 and the recorder task 24 continue to be performed and a replay task 
26 has been added. Input task 20 outputs the encoded stream to recorder task 24 via first 
connection 21. Replay task 26 outputs a delayed version of the input stream from storage 
device 18 (not shown) to decoder task 22 via a second connection 23. Decoder task 22 has 
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been switched back to the nonnal operating state, in which it reads video data from its input 
and uses that data to update its output 

111 the ^aratus of figure 1 tasks 20, 22, 24, 26 are executed by processing 
task elements 14a-c,f. Connections 21, 23 are implemented by FIFO conununication buffers 
5 1 6a-d. Tasks that are not required in a video mode are deactivated, for example by disabling 
tlie clock input to the processing task element in case of hardware implemented task 
elem^its, or by releasing resources used by these tasks in case of software implemented task 
elements, thus releasing system resom'ces for other use. 

When video mode control unit 15 switches the video display system between 
10 different video modes, the processing task elements 14a-c,f that are active before and after 
the mode switch are kept active (or at most suspended) through the mode switch, so that they 
will continue execution after the mode switch. The FIFO communication buffers 16a-d 
coupled to these continuingly active processing task elements 14a-c,f also remain active, but 
video mode control unit 1 5 controls buffer connection element 17 so that the coupling to or 
15 from some of the inputs or outputs of FIFO communication buffers 16a-d are switched to 
different processing task elements 14a-c,f. 

When a first one of processing task elements 14a-c,f has an input or output 
coupled, via a FIFO communication buiBfer 16a-d, to a second one of processing task 
elements 14a-c,f before the mode switch and to a third one of processing task elements 14a- 
20 c,f after the mode switch, iho second one of the of processing task elements 14a'C,f is 

decoupled &om this FIFO communication buffer 16a-d and the third one is coupled to this 
FIFO communication buffer 16a-d in its place during the video mode switch. Video data in 
the FIFO communication buffer 16a-d is left in place for later use. The second one of 
processing task elements 14a-c,f is deactivated (or its execution suspended) before being 
25 decoupled and the third one is activated or unsuspended after being coupled (or, if activated 
' earlier, it is supplied with a dunmiy "FIFO full" signal if its output will coupled to the FIFO 
communication buffer 16a-d or "FIFO empty" signal if its input is to be coupled). Thus, no 
data is lost during mode switching and no interruption of the video signals occurs. 

In the case of a switch from the live play mode of figure 2a to the pause mode 
30 of figure 2b, video mode switch control unit 15 sends a signal to the decoding task element 
14f to switch to tlie "freeze" state of operation, causing it to repeat a current frame. Video 
mode switch control then disconnects the FIFO communication buffer 16a-d that implements 
connection 21 from decoding task element 14f and connects it to a recording task element 
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14b. Subsequently, video mode switch control unit 15 activates the recording task element 
14b. 

In tiie alternative embodiment in which first connection 21 remains coupled to 
decoder task 22, video mode switch control unit 15 reconnects the output of input task 
5 element 14a to a second FIFO communication buffer 16a-d (optionally causing input task 20 
to suspend operation during reconnection) and connects recording task element 14b to that 
second FIFO conimimication buffer 16a-d before activating recording task element 14b. In 
this embodiment decoder task 22 does not have to be brought into a fireeze state. This 
simplifies design, but it has the disadvantage that some delay may occxu* during video mode 

10 switching because buffered data firom first comection 21 has to be processed before decoder 
task 22 outputs a fi-ozen image. 

In the case of a switch firom the pause mode of figure 2b to the time-shift 
mode of figure 2c, video mode switch control unit 15 coimects a replay task element 14c and 
the decoding task element 14f to a second FIFO communication buffer 16a-d that implements 

15 counection 23. Subsequently, video mode switch control unit 15 signals decoding task 
element 14f to return to the normal state of operation, in which it reads video data for new 
fi-ames from second FIFO commimication buffer 16a-d, and activates the replay task element 
14b.In the alternative embodiment where the first connection 21 remains, attached to 
decoding task 22, no new coxmection needs to be added of course. In this case video mode 

20 switch control unit 1 5 couples replay task element 14c to the FIFO communication buffer 
16a-d that implements first connection 21 and activates replay task element 14c. No signal is 
needed to switch decoder task element 14f back to the normal operating state in this case. 

In the case of a switchback from the time shift mode of figure 2c to the live 
play mode of figure 2a, video mode switch control unit 1 5 signals replay task element 14c to 

25 cease execution. Video mode switch control unit 15 commands recording task element 14b 
and replay task element 14c to cease execution. Then video mode switch control unit 15 
reconnects the input of decoding task element 14f firom the FIFO commimication buffer 16a- 
d that is connected to replay task element 14c to the FIFO conmiunication buffer 1 6a-d that 
implements first connection 21 to input task element 14a. Preferably a signal is sent to switch 

30 decoder task element 14f to the "fi-eeze" mode temporarily during reconnection. Afterwards 
the FIFO communication buffer 16a-d implementing connection 23 may also be released and 
its contents discarded. 

Altematively, input task element 14a may be coupled to the input of the FIFO 
conmiunication buffer 16a-d that couples replay task element 14c to decoder task element 14f 
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in the time shift mode. However, ttiis has the disadvantage that switching fiom time shift 
replay to live play occurs only with some delay, because decoder task element 14f has to read 
the old video data from the FIFO communication buffer 16a-d first. 

In the altemative embodimrat wherein decoder task 22 rmiains coupled to 
5 first connection 21, video mode switch unit 15 reconnects the input of decoder task 22 to the 
FIFO communication buffer 16a-d that implements second connection 23. It will be seen that 
thus, through switching firom the live play mode to the pause mode, to the time shift mode 
and back to the live play mode, a different one of FIFO communication buffers 16a-d is 
coupled to encoder task element 14f compared with the initial live play mode. This is a result 

10 of dynamic assignment of the FIFO communication buffers 16a-d. 

It wiU be appreciated that in this way no restart of any processing task 
elements 14a-f is needed when the element is active both before and after the video mode 
switch. Processing task elements 14a-f that are no longer needed are deactivated, releasing 
system resources for other use. Continued use FIFO communication buffers 16a-d before and 

15 after the video mode switch ensures continuity of the viewed video streams. It will also be 
appreciated that a similar implementation can be used for the reverse of the switches that 
have been described (i.e. switches firom time shift mode to pause mode, firom pause mode to 
live play mode and &Qm live play mode to time shift mode). 

Preferably, the processing task elements 14a-f are designed to select the time 

20 point where they cease to be active dependent on the progress of the video data being 

processed, so that the part of the stream after ceasing can be decoded without reference to 
earlier video data. In. an MPEG video signal, for example, I-frames, P-frames and B-fi:ames 
occur, the latter two types of firame being encoded as updates to oth^ frames. Closed 
"Groups Of Pictures" (GOP's) are identified in the video stream, so that the frames in each 

25 particular GOP are never encoded as updates to frames outside that particular GOP. 

In this case, when video mode control unit 15 signals to a processing task 
element 14a-f to suspend or cease operation as part of a video mode switch, the processing 
task element 14a-f finishes reading and/or writing of a GOP from or to a FIFO 
communication buffer 16a-d prior to actually ceasing or suspending operation. Thereupon the 

30 processing task element 14a-f signals back to video mode switch control unit 1 5 that tlie 

command has been executed. Subsequently, video mode switch control unit 15 signals buffer 
connection element 17 to couple the buffer or buffers involved to another processing task 
element 14a-f as required by the mode switch. After that video mode control unit signals the 
newly coimected processing task element 14a-f to start or resimie execution. 
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FIFO communication buffers 1 6a-d preferably provide for video data input, 
video data output and anpty/full signaling. The data width is not relevant to the invention but 
typically multibit datawords will be written and read. Each FIFO communication buffer 1 6a- 
d provides a full/not full signal back to the processing task element 14a-f that supplies data to 
S the buffer. If "full" is signaled and the processing task element 14a-f has data available the 
processing task element 14a-f suspends execution. Each FIFO communication buffer 16a-d 
provides an empty/not empty signal to the processing task element 14a-f that reads data from 
the buffer. If "empty" is signaled and the processing task element 14a-f needs to read data the 
processing task element 14a-f suspends execution. In the software implementation of 

10 processing task element 14a-f the processing task element 14a-f may use polling of the empty 
and Ml signals to determine when to resume operation, or event signaling or interrupt 
signaling may be used to resume operation. In the hardware implementation the empty and 
full signals may also be used as enable or disable signals. 

FIFO communication buffers 16a-d may be implemented as memory areas in a 

15 computer memory, the FIFO operation being controlled by software (keeping a pointer to the 
oldest unread data and tiie free location next to last written data, checking for full and empty 
conditions and setting flags accordingly or signaling those flags to coupled processing task 
elements 14a-f). Dedicated hardware FIFO buffers may be used as well. Similarly buffer 
connection element 17 may be implemented using software, in which case buffer connection 

20 element 17 may be integrated with FIFO communication buffers 16a-d in tiie form of 

pointers to processing task elements 14a-f to which respective FIFO communication buffers 
16a-d are coupled. Alternatively buffer connection element 17 may be integrated with 
processing task elements 14a-f in the form of pointers to FIFO communication buffers 16a-d 
to which respective processing task elements 14a-f are coupled. Mixtures of such pointer 

25 implementations are also possible. 

It will be appreciated that the invention is not limited to the specific 
embodiments that have been described. 



PHNL021390EPP 



9 



18.12.2002 



CLAIMS: 



1. A video viewing system, comprising: 

- an input foi receiving an incoming video stream; 

- an output for outputting an outgoing video stream; 



EPO-DGi 
1*12.2(52 



- a storage device for storing video data; 
5 - a plurality of FIFO communication buffers; 

* a processing system arranged for executing a plurality of tasks, including an input task 
coupled to the input, a decoding task coupled to the output, a recording task for recording on 
the storage device and a replay task for replaying from the storage device, the processing 
system being switchable between a plurality of video modes, including a live play mode in 

10 which the input task is coupled to the decoding task, a pause mode in which the input task is 
coupled to the recording task, and a time-shift mode in which the input task is coupled to the 
recording task and the replay task is coupled to the decoding task, the tasks being coupled via 
respective ones of the FIFO communication buffers, the processing system being arranged to 
switch between the video modes by reassigning connections of tasks to the respective ones of 

15 the FIFO communication buffers, keeping previous video data in the reassigned FIFO 

communication buffers, keeping tasks that are needed before and after a switch continuingly 
active and ceasing execution of tasks that are not in use after the switch. 

2. A video viewing system according to Claim 1, wherein the processing system 
20 is arranged to send a "freeze" signal to the decoder task upon a mode switch from the live 

play mode to the pause mode, to make the decoder task enter a freeze state in which the 
decoder task permanently outputs a current video frame. 

3 . A video viewing system according to Claim 2, wherein the input task is 

25 coupled to flie decoder task via a first one of the FIFO communication buffers in the live play 
mode, the processing system being arranged to decouple an input of the decoder task from 
an output of the first one of the FIFO communication buffers upon switching from the live 
play mode to the pause mode, to couple an input of the recording task to the output of the 
first one of the FIFO communication buffers and subsequently to activate the recording task. 
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4. A video viewing system according to Claim 3, wherein the processing system 

is arranged to couple an output of the replay task to an iirput of a second one of the FIFO 
communication buffers upon a video mode switch from the pause mode to the time-shift 
5 mode, to couple an input of the decoder task to an output of the second one of the FIFO 
communication buffers and subsequently to activate the replay task, the decoder task being 
switched back from the freeze state to a normal operating state. 

5^ A video viewing system according to Claim 1, wherein the input task is 

10 coupled to the decoder task via a first one of flie FIFO communication buffers in the live play 
mode, the processing system being arranged to couple a second one of the FIFO 
communication buffers between the input task and the recording task upon switching from 
the live play mode to the pause mode, an input of the first one of the FIFO communication 
buffers being disconnected from the input task and an output of the first one of the FIFO 
1 5 communication buffers remaining coupled to an input of the decode task. 

6. A video viewing system according to Claim 1 , wherein the input task is 
coupled to the recording task in flie tune shift mode via a first one of the FIFO 
communication buffers, the processing system being arranged to reconnect an mput of the 

20 decoder task to an output of the first one of the FIFO communication buffers upon a video 
mode switch from the time shift mode to the live play mode, deactivating the replay task and 
the recording task. 

7. A video viewing system according to Claim 1, wherein each task that is 
25 deactivated upon mode switching is arranged to delay deactivation upon mode switching 

until it has processed a closed group of pictures and written or read that group to or from one 
of the FIFO communication buffers. 

8. A video viewng system accordiug to Claim 1, wherein at least one of the 
30 replay task and recording task is implemented as a computer program running on a 

programmable computer^ so that resources used by said at least one of the replay task and 
recording task is released in a video mode or modes when said at least one of the replay task 
and recording task is not active. 
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9. A video viewing system accorfing to Claim 1. wherein at least one of the 
replay task and recording task is implemented in dedicated hardware, the dedicated hardware 
being switchable to a power saving state, the dedicated hardware for said at least one of fte 
replay task and recording task being released in a video mode or modes when said at least 

5 one of the replay task and recording task is not active. 

10. A method of producing an outgoing video stream for viewing, wherein a 
plurality of tasks are executed, including an itq>ut taskreceiving an incoming video stream, a 
decoding task outputting the outgoing video stream, arecording task recording on a storage 

10 device and a replay task replaying fix)m the storage device, 
the method oomimsing: 

- switching between aplurality of video modes, different subsets of the tasks or all of the 
tasks being active in different ones of the modes, the video modes including a live play mode 
in which the input task is coupled to the decoding task, a pause mode in which the mput task 

15 is coupled to the recording task, and a time-shift mode in which the input task is coupled to 
the recording task and the replay task is coupled to the decoding task, 

- using FIFO communication buffers to communicate between the tasks, switching between 
the video modes being realized by reassignmg connections of tasks to respective ones of the 
FIFO communication buffers, keeping previous data in Ihe reassigned FIFO communication 

20 buffers, keeping tasks that are needed before and aftac a switch continuingly active and 
ceasing execution of tasks ttiat are not m use after the switch. 



U. A conq)uter program product comprismg a program of computer mstructions 

for making aprogrammable computer perform the method of Claun 10. 
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ABSTRACT: 



{92) 

An outgoing video stream is produced in a plurality of video modes, including 
a live play mode in which an input task(20) is coupled to a decoding task (22), a pause mode 
in which the input task (20) is coupled to a recording task (24); and a time-shift mode in 
which the input task (20) is coupled to the recording task (24) and a replay task (26) is 

5 coupled to the decoding task (22). In different modes different subsets of the tasks (20, 22, 
24, 26) or all of the tasks (20, 22, 24, 26) are active. FIFO communication buffers (16a-d) are 
used to communicate between the tasks (20, 22, 24, 26). Switching between the video modes 
being realized by reassigning connections of tasks (20, 22, 24, 26) to respective ones of tiie 
communicaUon buffers (1 6a-d), keeping previous data in the reassigned FIFO 

1 0 communication buffers (16a-d). Tasks (20, 22, 24, 26) that are needed before and after a 
switch are kept continuingly active, so that they continue to read from or write to the FIFO 
communication buffers (16a-d) to which fliey are connected. Reassignment of connections to 
communication buffers (16a-d) preferably is limited to time points at transitions between 
transfer of closed Groups Of Pictures in the video streams. 
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FIG. 2c 



